15th International Conference on Cognition and Exploratory Learning in Digital Age (CELDA 2018) 


COLLABORATIVE TESTING STRATEGIES 
IN A COMPUTING COURSE 


Juliana de Melo Bezerra 
Instituto Tecnoldgico de Aerondutica — ITA, Sao José dos Campos — SP, Brazil 


ABSTRACT 


Beyond cognitive competence, other abilities are relevant to labor market nowadays, including critical thinking, self 
confidence, teamwork and communication. It is then important to introduce in the learning process practices that 
contribute to both the acquirement of knowledge and the development of soft skills. A prominent answer to this challenge 
is the use of the collaborative learning, where students work together to solve a problem or complete a task. In this paper, 
we present three strategies to implement collaborative testing in class. We also evaluate the application of such strategies 
in a computing course in order to provide evidences of their effectiveness compared to traditional tests. We found that 
collaborative testing contributed not only to improve students’ performance in tests, but have a positive impact on 
perceived learning and students’ satisfaction. 
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1. INTRODUCTION 


Technology is increasingly getting into people’s lives and becoming one of the keys of economy nowadays. 
Governments are trying to impulse its economy by developing the industry and using science to aggregate 
value to its products and services. Besides the importance of technical background, there are other skills that 
need to be developed during the education process. Wagner (2008) describe seven skills that companies 
expect students to develop, as follows: critical thinking and problem solving, collaboration across networks 
and leading by influence, agility and adaptability, initiative and entrepreneurialism, effective oral and written 
communication, accessing and analyzing information, curiosity and imagination. In other work, Anderson 
(2014) identified important skills such as flexibility, communication, cooperation, emotional maturity, and 
initiative. 

Education institutions must be prepared for this demand and for the necessary training of students, in 
order to respond to economic and social developments in the 21st century. According to Henderson and 
Dancy (2011), a barrier in the education process is the lack of knowledge about how to effectively use the 
available instructional ideas and strategies. Professors are investigating different ways to improve learning, 
considering the development of cognitive and non-cognitive skills, by using collaborative activities during 
the course. Student-centered instructional strategies are proving to be more effective in improving students’ 
conceptual understanding, knowledge retention, and attitudes about learning than traditional methods that do 
not include student participation (Kober, 2015). 

Collaborative testing is a method in which students work together while taking an evaluative exam. 
Testing is a means to identify whether teaching is effective and how well the student comprehends subjects. 
Important outcomes, related to enhancement of students’ learning, are been generated through investigations 
of the use of collaborative testing in distinct contexts (Bjornsdottir et al, 2015; Cantwell et al, 2016). For 
instance, Willey and Gardner (2012) reported the capacity that collaborative frameworks have to develop an 
effective and integrated learning experience for students. Shen et al (2008) found that collaborative testing 
enhanced interactions among students and increased the perceived learning. In the present paper, we present 
three collaborative testing strategies and discuss relevant results associated to their application in a 
computing course. 
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2. COLLABORATIVE TESTING: STRATEGIES AND APPLICATION 


In this section, we explain the selected collaborative testing, and present the experiment design and results. 


2.1 Collaborative Testing Strategies 

Three collaborative testing strategies were designed, as follows: “Question Revision”, “Question Discussion” 
and “Pair Work”. Figure la shows the scheme of the “Question Analysis” strategy. The strategy consists of 
using knowledge acquired during a test part to help solving other parts of the test. The steps of the strategy 
are: 1) Each student answer question A individually; 2) Instructor separates students into groups in order to 
discuss their question and understand their mistakes, without changing their answer; 3) Students separate the 
groups and solve question B individually, which uses the same idea of question A but in a more complex 
context. To get the expected result it is important that students discuss question A and try to figure out the 
answer for it based on what each student wrote and on the discussion itself. 

Figure 1b present the scheme of the “Question Discussion” strategy. The strategy tries to use the power of 
initial discussions to accelerate the reasoning of students. The steps of the strategy are: 1) Instructor forms 
groups and give them some time to discuss the main idea of Question A; 2) Students separate from the 
groups and solve question A individually, but using the discussed ideas; 3) Students solve question B 
individually, which uses the same idea of question A but in a more complex context. The purpose of this 
strategy is just to accelerate the rationality behind the question, and not have a group question. 
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(a) (b) (c) 
Figure 1. Collaborative testing strategies. a) Question Revision. b) Question Discussion. c) Pair Work 


The scheme of the “Question Discussion” strategy is shown in Figure lc. With this strategy, students 
have the chance to correct some mistakes during a test. The strategy uses the power of a peer review to give 
students the possibility to develop their critical thinking and discussion skills. After instructor separates the 
class into pairs, the steps of the strategy are: 1) Each student of a pair receives one different question and 
solve it individually; 2) After a stipulated time, students change tests with their pairs and start the review 
process; 3) Each pair discusses the best answer for each question and hand over the test. This strategy does 
not require dependences between questions, but it is important to define pairs to avoid them being uneven. 


2.2 Experiment Design 


The experiment was taken place at an engineering school, during the course of “Data Structure and 
Algorithms” from the department of Computer Science. We applied the three collaborative tests in a class of 
46 students. An important part of the experiment was the selection of subjects for each collaborative strategy. 
We used “arrays and recursion” topic with “Question Revision” strategy, “linked list” topic with “Question 
Discussion” strategy, and “queue and stack” topic with “Pair Work” strategy. To evaluate the effectiveness of 
the proposed strategies, we considered the following aspects: students’ performance, perceived learning, and 
satisfaction of students. The first one was considered to test the concept that students would have better 
grades if tests involve some collaborative activity. The second one was to understand if students taking 
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collaborative tests would have a better perception of learning than taking traditional exams, whilst the last 
one is to have a sense if students liked the proposed activities. 

In order to evaluate students’ performance with “Question Revision” and “Question Discussion” 
strategies, we divided the class into two random groups, each one with half of the class. One group (called 
experiment group) used the collaborative strategy, while the other group (called control group) used 
traditional test format. For the “Pair Work” strategy, the whole class participated in the collaborative activity 
(configuring the experiment group) and the control group was a class in the previous year that performed a 
traditional test with the same questions. 

The aspects of perceived learning and students’ satisfaction were evaluated based on the perception of 
students who participated in the collaborative activities. Evaluation sentences are shown in Table 2. 
Sentences L1 to L6 were regarding perceived learning, while sentences S1 and S2 were about students’ 
satisfaction. Students used a 5-point likert scale (with the values: strongly agree, agree, neutral, disagree, and 
strongly disagree) to assess sentences L1 to S1. The last sentence S2 had “yes” or “no” as possible answers. 
Sentences LI to S1 were assesses by all students in the beginning of the course, considering their experience 
with traditional tests. All sentences were evaluated after each collaborative test but only by students in the 
experiment group, who really experienced the collaborative strategy. 


2.3 Results 


Performance results for each collaborative test are shown in Table | using the mean of grades (0 to 10.0) of 
students in experiment group (with collaborative test) and control group (with traditional test). Regarding 
“Question Revision” strategy, we observed that experiment group had lower performance in Question A 
compared to control group. It was not a problem, since the strategy include a step to discuss Question A but 
without changing it. The discussion was expected to positively influence the grades in Question B (more 
complex than Question A), which was in fact observed with a 10% of performance improvement. Notice that 
grade mean in Question B was similar in both groups, however the discussion really helped the experiment 
group to overcome their difficulties and reach better results. 


Table 1. Performance of students 


______Strategy ____—<Grouptype Question A __QuestionB__ 
Question Revision aes rr a 7 
Question Discussion Caines re a Bet 
por Wok mimo 


Table 2. Sentences and results about perceived learning and students’ satisfaction 


Id Sentence Question Question Pair 
Revision _ Discussion Work 

L1 The test positively contributed to my learning process. 13% 6% 16% 

L2 The test made me realize that I know the subject studied. 16% 15% 15% 


L3 With the test, it is possible to learn from answers and 


9% 11% 13% 

comments of my peers. 
L4 The test contributed to develop my critical thoughts. 39% 11% 35% 
LS The test contributed to develop my analysis capacity. 6% 10% 12% 
L6 The test contributed to develop my self confidence. 71% 12% 6% 
S1 I feel anxious doing the test. -13% -17% -6% 
S2 I would like to do again this kind of test 87% 78% 719% 


Regarding “Question Discussion” strategy, the previous discussion about Question A, before solving it, 
contributed to a better perfomance of experiment group. The performance of experiment group was 11% 
higher compared to control group. In Question B, the difference of 3% between the two groups was not 
expressive enough to conclude that it was due to the discussion or if it is just a normal deviation in the class. 
So it was not clear is if discussing Question A helped them to figure out the solution of Question B. In “Pair 
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work” startegy, we observed a performance improvement of approximately 25% in each question. It was 
expected since the strategy allowed students to change their answers after using the contribution of their 
peers. Results of the evaluation of perceived learning and students’ satisfaction are presented in Table 2, with 
the percentage representing how much more the collaborative test is higher than the traditional test (assessed 
in beginning of the course considering the previous experience of students). The result of S2 sentence is a 
percentage that represents the amount of students that answered “yes” after performing the collaborative test. 
Analyzing the results, we noticed that all sentences in all tests had higher scores compared to traditional tests, 
except in case of SI sentence. We expected that students would feel more anxious during traditional tests; 
however students were apprehensive in collaborative test because they had never experience that and they do 
not know if it could negatively affect their grades. Regarding perceived learning, students acknowledged that 
collaborative tests were better for their learning, especially for developing critical thinking and for assessing 
the acquired knowledge. The majority of students, approximately 80%, indicated that they would like to have 
other opportunities with collaborative testing. 


3. CONCLUSION 


We presented three collaborative testing strategies and applied them in a computing course. Based on the 
proposed strategies, adapted versions and even new strategies can emerge, which are extremely valuable for 
educational community. The application of collaborative testing is possible even in the presence of other 
forms of evaluation; in our case, we also have traditional tests and practical laboratories. It may be difficult to 
select the strategy more suitable to the current context, given aspects as definition of questions and 
organization of class time. The evaluation of strategies is also a challenge, since it depends on several factors 
including nature and difficulty of questions, time given for each test, context and academic demands of 
students, and the way the strategy is applied. Considering our results, the approval of the usage of 
collaborative tests by students, aligned to the students’ perception that such tests were more effective than 
traditional ones in term of learning, are powerful evidences of the potential of collaborative testing. The 
collaboration itself brings the opportunity to students practice other abilities, such as to expose ideas, to 
negotiate, and to convince others. Collaborative testing, in turn, contributes not only help students to learn 
more about technical subjects, but also to improve their non-cognitive skills as communication and 
teamwork. As future work, we intend to apply distinct collaborative testing strategies and to investigate other 
kinds of effectiveness evaluation. 
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